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FACTOR COVARIANCE ANALYSIS IN SUBGROUPS* 



Roger Pennell 

Educational Testing Service 
Abstract 

The problem considered is that of an investigator sampling two or 
more correlation matrices and desiring to fit a model of the form 

p 

R = Pd> P' + U. . Here the factor pattern matrix, P , is assumed to 
11 i 

be identical across samples and we need to estimate <j>^ and . 

A flexible, least squares solution is worked out and illustrated with 
an example. 



*The author wishes to acknowledge the impetus provided by an unpublished 
paper by Samuel Messick detailing the problem solved in the present paper. 



FACTOR COVARIANCE ANALYSIS IN SUBGROUPS 



I. Introduction 

An investigator frequently finds himself confronted with data from two 
or more groups. The groups frequently arise (l) by longitudinally or cross- 
sectionally measuring samples of subjects on the same variables or (2) by 
partitioning a sample into subgroups using an explicitly defined selection 
variable. An example of the first category may involve measuring all students 
from the 7th, 9th, and 11th grades on a set of variables or measuring a group 
of students in year v and again in years v + 2 and v + 4 . An example 
of the second may occur by dividing a sample into two groups based on sex or 
into multiple groups based on the kind of teaching philosophy they have been 
exposed to, etc. 

The investigator may want to compare the factorial composition of the 
groups using factor analysis. What is most frequently done is to factor analyze 
the respective correlation matrices and use an orthogonal or oblique Procrustes 
procedure to rotate the two initial solutions to a position of maximum congru- 
ence. This is surely the least defensible approach, theoretically, and in 
practice often produces ambiguous results (e.g., Meredith, 1964a). Indeed 
for purposes of orthogonal factor matching, one is embracing a very strong 
model, to wit diagonal factor covariance matrices within groups and common 
factor patterns across groups . 

Meredith (1964b) has proposed a general solution for the above situation. 

He assumes the subgroups arise by multivariate selection on known or unknown 
selection variables. After finding an orthogonal factor pattern for each 
subgroup, each pattern is rotated to be as similar as possible while per- 
mitting the factor covariance matrices to vary. 



3 



- 2 - 



There are other models which attempt to synthesize the results of two 
factor analyses or, in fact, to produce factor-analytic results directly for 
two groups of subjects or batteries of tests. Such a model was proposed by 
Corballis and Traub (1970) to assess the degree of change between two sets of 
factors measured on two occasions on the same set of subjects. Their procedure 
estimates the within-occasion factor matrices and then finds rotation matrices 
and measures of factor similarity which together approximate the between- 
occasion correlations. This model requires very strong assumptions, namely, 
near symmetry of the between-occasion correlation matrix. As this matrix 
departs from symmetry, the rotation matrices necessarily impose greater 
disparity on the two within-occasion matrices , thus rendering the presumption 
of same factors across occasions less compelling. Corballis and Traub (1970) 
note this potential difficulty, but do not relate it to the between-occasion 
correlation matrix. 

Interbattery factor analysis (Kristof, 1967; Tucker, 1958) is another 
method of investigating the fact or- analytic structure of two batteries of 
tests. Kristof' s (1967) model hypothesizes a set of factors common to both 
batteries and a set unique to each battery. This method is important because 
it finds variables in both batteries which mark the same factor and thus 
provides continuity between batteries composed of possibly different measures. 
If the same measure, or set of measures, is included in each battery, differ- 
ences in the observed pattern of factor loadings may be due to the factor 
measuring a different trait in one battery or may reflect differing factor 
covariances (other than orthogonal) . 

We shall propose a somewhat different model, one that postulates virtually 
identical factor patterns across groups, but permits the factor covariance and 
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factor structure matrices within groups to vary. Namely, the model assumed to 
hold in each group is 

(1) Z. = PX. + U.Y. 

' ' l l l l 5 

where Z. is a row centered, observed score matrix of n tests by N 

l 

observations, P is an n x q factor pattern matrix, X^ is a q x N 

common factor score matrix, U is an n x n diagonal matrix of unique 

standard deviations, and Y^ is an n x N matrix of unique factor scores. 

If we assume X.Y! = 0 and a suitable scaling on X. , the factor-test 
ii l 

covariance matrix (structure matrix; Harman, 1967) for each group is 

(2) S. = P*. 

l i 

where cf>. = X.X! , the factor covariance matrix. Translated into practical 

ill * 

terms the model in (l) and (2) says that the one way in which we can opera- 
tionally insure the enduring nature of general psychological traits (factors) 
is by requiring the same linear combination (P) of the factor scores to 
reproduce the common portion of the observed scores. The pattern matrix 
( P ; Harman, 1967) is typically the matrix factor analysts use to under- 
stand the factors they obtain, and is thus one way to conceptualize a set of 
enduring psychological traits. The traits may change in the way they covary 
with the observed tests (S^) and with other traits ( <j> ) , but at least we 
are certain of our ground in calling them the same factors. The model differs 
from Meredith's (1964b) in that we assume factorial invariance to begin with. 

II. Method 

Assume we are given p covariance matrices Ch , i = 1,2,... ,p , 
among n tests or measures , with observations made on each measure in 
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4- Vi 

the i set. We wish to estimate a matrix P such that 
(3) R. = P<J).P' + UT 

l i l 

holds in each subgroup, i here, R. is an n x n correlation matrix, 

2 

d> is q x q , and U. is an n x n diagonal matrix of unique variances. 

' i i 

Apparently, the most stable estimate available would result from pooling the 
group covariance matrices as 





i 

i 




( 4 ) 




(N 



1)C. 

1 
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where N = ZN , and scaling C to a correlation matrix as 
i 

(5) R = DCD , 



where D is diagonal and contains the reciprocals of the square roots of the 
diagonal part of C . For R we are considering the model 

(6) R = P«j>P' + U ? 

At this point we merely need to estimate and factor R - using any 

number of available routines . Assume 

(7) R=AA 2 A' +U 2 



is such a factorization with A orthogone.1 by columns and of rank q < n . 
Let T be any satisfactory oblique, q x q , transformation matrix. A 
suitable estimate of P is given by 

(8) p = aa(t' r 1 

(Harman, 1967, P* 284). 

In order to insure a common scaling in each subgroup we use D from 
(5) as 
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(9) 



R. = DC . D 
1 l 



It should be noted that R_^ is not a correlation matrix for the subgroup, 

"fch 

but, rather, the i covariance matrix with the population scaling imposed. 

The problem now is to estimate <)k from the model 

(10) R. = Pd>.P' + U 2 

1 1 

2 

This can be done with no knowledge of the U by observing that the off- 
diagonal elements of R. are functions of P^P 1 only. Therefore , let us 
consider the n(n - l)/2 unique linear equations for these off-diagonal 
elements . They can be written in general equational form as 



(id 



r ij 



i = 1 ,2 , . . . ,n-l 



A JL + p±mP J *• 1 



j = i+l,i+2 , . . . ,n 



+ ^ hk^Ak + e u 



Let 0 represent a column vector of the n(n - l)/2 elements r. . , l ; a 

l J 

column vector of the unknown <t>„ , and B the n(n - l)/2 x q(q + l)/2 matrix 

x,m 

of coefficients. Equation (ll) can then be written in matrix notation as 



(Ha) 



0 = + E 



O 

ERIC 



where E is a vector of errors with the same order as 0 . Then clearly the 
sum of squared errors, E f E , is minimized by taking 

( 12 ) 



5 = (B , B)” 1 B , e 



A A 

Now we need only array the elements of £ in the symmetric matrix <j>^ 

2 

and produce a least squares estimate of FL and IL . Note that 



(13) 



U? = Di ag ( R . - Pdj.P' ) 
1 11 



( 



_r,_ 



and 

(lit) R. = P<(>.P’ + U 2 

111 

III. Computational Considerations 

Since B is of order n(n - l)/2 x q(q + l)/2 and we wish to invert 
B'B , it is clear that n(n - l)/2 must be at least as la.rge as q(q + l)/2 
which implies that q <_ n - 1 . This clearly posses no serious problems, since 
the goal of factor analysis is to isolate factors which are "substantially" 
smaller in number than the number of observed measures. 

The restriction that q n - 1 does not necessarily imply that B'B 
is of full rank and thus invertible as needed to compute a solution. Since 
B is computed from elements of a rank q matrix one might be initially 
suspicious about the full column rank assertion of B . However, by 
appealing to rather simple notions of no column linear dependencies existing 
in B , it can be shown that B is of full column rank if and only if P is. 

The most serious limitations of the solution is computer capacity to 
invert B'B . Problems with four or five factors, or even 10 or 15 present 
no difficulty, but a 20-factor problem, say, simply cannot be handled since 
we need to invert a 210 x 210 matrix. Computer storage problems, however, are 
almost completely a function of the number of factors; a fairly large number 
of tests can easily be handled. 

A problem not typically treated involving the estimation of covariance 
matrices is the restriction that the solution matrix be Gramian. The general 

A 

solution presented above in no way implies that is Gramian. Along these 

lines an interesting feature of the model equation (ll) is that it can be 
rewritten to reflect various hypotheses. An example would be 
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(15) 



r . 




+ E Z 
£=1 m=£+l 



q q 




i = 1,2, .. . ,n-l 



j = i+l,i+2,. . . ,n 



which differs from (ll) only by assuming that <j>^, has unities on the 



diagonal and is thus a correlation matrix. This is not necessarily a very 
interesting hypothesis, but it does indicate that the model is fairly flexible. 
If we should wish, we can control departures from Gramian form by observing 
them and reformulating the model as exemplified in (15). In one set cf data 
actually analyzed a small negative value occurred for one of the diagonal values 
of <f> . In this case it seemed reasonable to assume that this factor was simply 
not operating at all in this group; the data for the group in question were 
reanalyzed assuming zero variances and covariances for this factor. 

A flexible computer program to perform the above analysis has been written 
for an IBM/360 (Pennell, 1970). The program is monitored by a driver program 
which allocates storage to the data in hand in such a way as to minimize the 
portion of the machine needed and thus minimize costs. The program also 
includes an automatic reanalysis feature upon detection of negative variances 
in <p . 



In order to illustrate the procedure, data from the Holzinger and 
Swineford (1939) monograph were used. Scores on 2b cognitive tests were 
obtained from seventh and eighth grade subjects from two schools. There are 
a variety of ways in which the data could be broken down; however, for our 
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purposes the total sample was divided into male and female data. Starting 
from the raw data the correlation matrices in Table 1 were computed where 
N = 146 for males and N = 155 for females . 



Insert Table 1 about here 



The pooled correlation matrix was factored using principal factor analysis 
and iterated communalities for four factors. The final solution was then 
rotated using direct oblimin (Harman, 1967) which produced a good fit to 
various published solutions. The factors are usually identified along the 
lines of a spatial relations factor (i), a verbal factor (II), a perceptual 
motor speed factor (ill), and a memory factor (IV). The factor intercorrelation 
matrix for the pooled sample is presented in Table 2, while subgroup factor 
correlation matrices and factor variances are presented in Table 3. 



Insert Tables 2 and 3 about here 

The fit of the model to the male-female data is quite good. A rough index 
of fit is E'E/B'B which gives the ratio of the sum of squared error to the 
sum of squared parameters to be fitted. For the males this index is .052, and 
for the females it is .031. 

Two features of the data are worthy of note. First, the generally higher 
factor intercorrelations for the females suggest somewhat ‘‘ess differentiation 
of the traits measured by the factors. Notable, as well, is the significantly 
lower variance for the verbal factor for the females. Even though scores on 
the measures constituting this factor tend to be higher for females, this 
factor is a good deal less important in explaining differences in the original 
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Table 2 

Pooled Factor Correlation Matrix (N = 501) 



1.0 


.245 


.249 


.404 




1.0 


.268 


.585 






1.0 


.267 








1.0 



O 

ERJC 
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Table 3 

Factor Correlations and Factor Variances for 
Males (N = 146) and Females (N = 155) 











Males 








-- 


.154 


.167 


•327 






.354 
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.179 


.341 


Females 




.321 


.383 


-- 


.192 






.466 


.440 


.326 


-- 


Factor 


M 


.886 


1.264 


1.035 


.954 


Variances 


F 


1.107 


.752 


.969 


1.044 
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